Validating research performance metrics against peer rankings
نویسنده
چکیده
A rich and diverse set of potential bibliometric and scientometric predictors of research performance quality and importance are emerging today — from the classic metrics (publication counts, journal impact factors and individual article/author citation counts) to promising new online metrics such as download counts, hub/authority scores and growth/decay chronometrics. In and of themselves, however, metrics are circular: They need to be jointly tested and validated against what it is that they purport to measure and predict, with each metric weighted according to its contribution to their joint predictive power. The natural criterion against which to validate metrics is expert evaluation by peers; a unique opportunity to do this is offered by the 2008 UK Research Assessment Exercise, in which a full spectrum of metrics can be jointly tested, field by field, against peer rankings.
منابع مشابه
The Association between Four Citation Metrics and Peer Rankings of Research Influence of Australian Researchers in Six Fields of Public Health
Doubt about the relevance, appropriateness and transparency of peer review has promoted the use of citation metrics as a viable adjunct or alternative in the assessment of research impact. It is also commonly acknowledged that research metrics will not replace peer review unless they are shown to correspond with the assessment of peers. This paper evaluates the relationship between researchers'...
متن کاملReassessing the Goals of Grammatical Error Correction: Fluency Instead of Grammaticality
The field of grammatical error correction (GEC) has grown substantially in recent years, with research directed at both evaluation metrics and improved system performance against those metrics. One unvisited assumption, however, is the reliance of GEC evaluation on error-coded corpora, which contain specific labeled corrections. We examine current practices and show that GEC’s reliance on such ...
متن کاملWhat is the effect of country-specific characteristics on the research performance of scientific institutions? Using multi-level statistical models to rank and map universities and research-focused institutions worldwide
Bornmann, Stefaner, de Moya Anegón, and Mutz (in press) have introduced a web application (www.excellencemapping.net) which is linked to both academic ranking lists published hitherto (e.g. the Academic Ranking of World Universities) as well as spatial visualization approaches. The web application visualizes institutional performance within specific subject areas as ranking lists and on custom ...
متن کاملMetrics and Rankings: Myths and Fallacies
In this paper we provide an introduction to the field of Bibliometrics. In particular, first we briefly describe its beginning and its evolution; we mention the main research fora as well. Further we categorize metrics according to their entity scope: metrics for journals, conferences and authors. Several rankings have appeared based on such metrics. It is argued that these metrics and rankings...
متن کاملR2-B.2: Portable, Integrated Microscale Sensors (PIMS) for Explosives Detection
This project seeks to develop portable, integrated, microscale sensors (PIMS), founded upon chemomechanically-induced response bifurcations, which are suitable for vapor-phase explosives detection. In prior work by the investigators, these sensing system were shown to yield superior performance metrics (i.e. false positive/negative rates and sensitivity metrics) in laboratory environments withi...
متن کامل